Stochastic Shortest Path with Energy Constraints in POMDPs
نویسندگان
چکیده
We extend the traditional framework of POMDPs to model resource consumption inducing a hard constraint on the behaviour of the model. Resource levels increase and decrease with transitions, and the hard constraint requires that the level remains positive in all steps. We present an algorithm for solving POMDPs with resource levels, developing on existing POMDP solvers. Our second contribution is related to policy representation. For larger POMDPs the policies computed by existing solvers are too large to be understandable, an issue particularly pronounced in POMDPs with resource levels. We present a procedure based on machine learning techniques that extracts important decisions of a policy and outputs its readable representation.
منابع مشابه
Stochastic Shortest Path with Energy Constraints in POMDPs: (Extended Abstract)
We consider partially observable Markov decision processes (POMDPs) with a set of target states and positive integer costs associated with every transition. The traditional optimization objective (stochastic shortest path) asks to minimize the expected total cost until the target set is reached. We extend the traditional framework of POMDPs to model energy consumption, which represents a hard c...
متن کاملNew Grid-Based Algorithms for Partially Observable Markov Decision Processes: Theory and Practice
We present two new algorithms for Partially Observable Markov Decision Processes (pomdps). The first algorithm is a general grid-based algorithm for pomdps with theoretical optimality guarantees. The other algorithm is for the subclass of problems known as Stochastic Shortest-Path problems in belief space. Both algorithms are optimal and robust with respect to a novel robustness criterion that ...
متن کاملCockpit Crew Pairing Problem in Airline Scheduling: Shortest Path with Resources Constraints Approach
Increasing competition in the air transport market has intensified active airlines’ efforts to keep their market share by attaching due importance to cost management aimed at reduced final prices. Crew costs are second only to fuel costs on the cost list of airline companies. So, this paper attempts to investigate the cockpit crew pairing problem. The set partitioning problem has been used for ...
متن کاملDynamic Multi Period Production Planning Problem with Semi Markovian Variable Cost (TECHNICAL NOTE)
This paper develops a method for solving the single product multi-period production-planning problem, in which the production and the inventory costs of each period arc concave and backlogging is not permitted. It is also assumed that the unit variable cost of the production evolves according to a continuous time Markov process. We prove that this production-planning problem can be Stated as a ...
متن کاملThe Lagrangian Relaxation Method for the Shortest Path Problem Considering Transportation Plans and Budgetary Constraint
In this paper, a constrained shortest path problem (CSP) in a network is investigated, in which some special plans for each link with corresponding pre-determined costs as well as reduction values in the link travel time are considered. The purpose is to find a path and selecting the best plans on its links, to improve the travel time as most as possible, while the costs of conducting plans do ...
متن کامل